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2 Introduction 

The Arkansas Leadership Academy (ALA) was established in 1991 and is a nationally recognized statewide 
partnership that includes 15 universities, 9 professional associations, the Arkansas Departments of Education, 
Higher Education, Career Education, and several other government and business agencies. In 2011, there 
were 49 partners involved collaboratively with the ALA (Arkansas Leadership Academy, n.d.). According to 
the ALA’s website: 


2.1 

The Academy, through the use of research and best practices, designs creative and innovative approaches 
to establish learning communities in public schools by developing human resources and by modeling and 
advocating collaboration, support, shared decision making, team learning, risk taking, and problem solving. 
Partners commit to changing their organizations to support system improvement. (About Us section, para. 
2 ) 

Embedded in the broad range of programs provided by the ALA, the Master Principal Program (MPP) 
specifically focuses on the development of principal leadership throughout the state of Arkansas (Arkansas 
Leadership Academy, 2012). The Arkansas Leadership Academy Master Principal Program (ALA MPP) is 
a multi-phase program established by the Arkansas 84 th General Assembly to improve school performance 
through effective leadership development (Act 44, 2003). 

The ALA MPP design focuses on five performance areas in order to improve principals’ effectiveness in 
school leadership, and subsequently, increase student achievement. These five performance areas were used 
to frame the curriculum and portfolio assessment for principals. The performance areas were identified after 
an extensive review of the research addressing leadership traits and characteristics found in and outside of 
the field of education (Elliott & Morledge, 2005). In addition, the performance areas were aligned to the 
Interstate School Leaders Licensure Consortium Standards for School Leaders (ISSLC). The five performance 
areas are as follows: 

• PAl: Creating and living the mission, vision and beliefs 

• PA 2: Leading and managing change 

• PA3: Developing deep knowledge about teaching and learning (academic rigor and relevance; effective 
teaching and learning) 

• PA4: Building and maintaining collaborative relationships 

• PA5: Building and sustaining accountability systems 

Applicants for Phase 1 of the ALA MPP apply for a limited number of openings annually and are selected 
from across the state to ensure a balance within each cohort group by grade level, geography, ethnicity, 
and years of experience. Selection for Phase 1 is based on meeting four eligibility criteria: 1) be a licensed 
principal in Arkansas, 2) have at least one year of experience as a principal including the current school 
year, 3) be a current principal in Arkansas, and 4) have the permission and support of the superintendent 
(Arkansas Department of Education, 2007). 

Participants in the ALA MPP Phase 2 must meet the Phase 1 eligibility requirements and must complete 
a portfolio demonstrating sufficient evidence for each of the five performance areas based on performance 
indicators detailed in the ALA MPP rubric (see Appendix A) and measured by the conceptual scoring frame¬ 
work (see Appendix B). The rubric is designed to differentiate applicants’ readiness for further development 
as a Master Principal. Applicants who complete Phase 1 of ALA MPP and receive an average score between 
2 and 3 points out of five for each of the five domains of the rubric are considered for MPP Phase 2. Similarly, 
applicants who complete Phase 2 of ALA MPP and receive an average score between 3 and 4 points out of 
five domains are considered for MPP Phase 3. Designation as a Master Principal is the final Phase of the 
program following successful completion of Phase 3 and a rigorous selection and review process. 
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3 Study Purpose 

The purpose of this study was to investigate whether the MPP approach to developing leadership knowledge 
and skills of sitting principals had an effect on principals’ ability to apply new knowledge and skills to 
improve their schools’ processes. The authors, (i.e., university educational leadership faculty and ALA staff), 
hypothesized the use of reflective practice and peer learning networks for engaging in program curriculum 
would result in changes in personal and organizational performance as a result of principals’ continued 
participation in the Arkansas Leadership Academy’s Master Principal Program. Furthermore, this study 
examined the nature of reflection practiced by the participants through their written narratives. The primary 
delivery methods of ALA MPP leadership development are reflective practice coupled with state-wide peer 
learning supports (RPPLS). The study involved an initial approach using quantitative analysis to examine 
the hypothesis that principals participating in the Master Principal Program were able to produce improved 
results on their portfolio assessments and in their school’s performance. In addition, a qualitative analysis 
of the text of the portfolio entries was used to extend the inquiry into the reflective practice of the MPP 
participants. 

4 Theoretical Framework 

For change to occur, peer interaction in the problem solving process is required whether it is within a single 
organization or between peer organizations (Fullan, 2008). Organization and system leaders have to move 
from operating autonomously in isolation to engaging in meaningful dialogue, shared problem-solving, and 
joint learning ventures that are only possible through effective communities of practice (Wagner & Kegan, 
2006). Hall and Hord (2011) categorize change as “a process through which people and organizations move 
as they gradually learn, come to understand, and become skilled and competent in the use of new ways” (p. 
8). Changes in day-to-day practices are implemented by individuals as a result of learning and may take 
three to five years to be fully realized to the highest levels of implementation (George, Hall, & Uchiyama, 
2000; Hall & Loucks, 1977; Hall & Rutherford, 1976 as cited in Hall & Hord, 2011). Therefore it is expected 
that leaders will demonstrate a meaningful change in leadership knowledge and skills as a result of long 
term participation in reflective practice and peer learning support networks through the Master Principal 
Program (see Figure 1). 


* formal Preparation 
and Pre-Service 
Training 


Knowledge In- 
Action 


/ 

Reflection-ln- 

Action 

■ ALA MPP In-Service 
Training Reflective 
practice and peer 
learning support 
nelworfc* 
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In personal! and 
organizational 
performance 


Outcome 


Figure 1. Theory suggesting that when principals experience the ALA MPP, thus being immersed in 
reflection-in-action, there is meaningful change in their personal performance and the performance of their 
organization. 

A majority of school principals complete formal training, usually in the form of a university-based prepa¬ 
ration program. While the professional preparation to become an administrator assists in defining roles and 
gaining entry into the profession, continued professional learning once in the profession has been recognized 
as a critical component to the establishment of effective professional practice (Houle, 1980). The nature of 
this professional learning is critical to the success of the practitioner. From the field of nursing, Williams 
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(2001) offers: 


4.1 

The situations professional nurses find themselves usually are so complex that the notion of routine practice 
should be precluded. To engage in competent nursing practice, nurses need to consider the contextual vari¬ 
ables in each interaction, mindful that each interpersonal encounter is to some extent unique and that there 
is usually more than one desired outcome.... They might need to question the routine, explore alternatives 
and transform previous ways of understanding, (p. 30) 

It could easily be argued that this same line of reasoning about how nurses need to regularly reflect on 
the job should be applied to principals and other school leaders. 

In examining how individuals continue to learn once in the job, adult learning theory and transformative 
learning theory illuminate specific principles that are paramount to the continuing education of professionals. 
The constructs of adult learning theory (Cervero, 1988; Knowles, 1980, 1992) and transformative learning 
theory (Mezirow, 1997) are embedded in the ALA MPP leadership development program with emphasis on 
the principles of reflective practice (Schon, 1983) and peer learning (Boud, 1999). While not de-valuing the 
importance of the knowledge acquired through formal preparation programs (most often resulting in what 
is known as knowing-in-action) , the focus of the ALA MPP is on the continuing training of the principal 
(which results in reflection-in-action). The difference between knowing-in-action and reflection-in-action 
(Cervero, 1988; Sch"n, 1983) is fundamental to this theoretical framework, and thus, should be addressed 
before encountering the literature on reflective practice and peer learning. 

4.2 Reflection in Action vs. Knowing in Action 

Knowing-in-action is evident when a professional practitioner uses knowledge that they have accrued from 
prior experience and applies it to a current situation (Schon, 1987). For many new principals, the knowledge 
discharged in knowing-in-action might often be based on the theoretical knowledge that has been learned 
through professional socialization sources or formal pre-service training experiences. For example, school 
principals typically go to a university educational leadership program to learn the theories of educational 
leadership and how to apply it to a practice situation. With this knowledge, there are preconceived assump¬ 
tions that are made as to how approach a particular problem (Mezirow, 1997). While making decisions and 
solving problems using prior knowledge and theoretical frameworks may assist in the success of the daily 
routines of being a principal, seldom does it lead to significant transformative change in the performance of 
the individual or the organization (Mezirow, 1997). 

Reflection-in-action, on the other hand, addresses the learning of knowledge based on an individual 
learner’s experience in the profession. As the individual’s experience is ongoing for the duration of time they 
are in the profession, reflection-in-action also is continuous and is seen as a cornerstone to producing new 
knowledge that assists the practitioner in their everyday work. Sch“n (1987) explained: 


4.2.1 

Reflection-in-action has a critical function, questioning the assumptional structure of knowing-in-action. We 
think critically about the thinking that got us into this fix or this opportunity; and we may, in the process, 
restructure strategies of action, understandings of phenomena, or ways of framing problems. (Schon, 1987, 

p. 28) 

Reflection-in-action leads to learning that addresses the contextual aspects of the work. Ongoing reflective 
practice and peer learning may or may not use prior knowledge to anchor thoughts, generate new ideas, or 
solve problems. It is the influence of a particular situation, organization, or culture that leads to the 
questioning of the assumptions that are present in broad-stroke theoretical concepts often found in knowing- 
in-action (Schon, 1987). 

Professional knowledge should be acquired from both knowing-in-action and reflection-in-action (Cervero, 
1988). Having a base of theoretical knowledge allows professional practitioners to have a platform to continue 
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to grow in their learning as well as their practice through experimenting and questioning the status quo. 
It could also be argued that the contextual nature of education and the often spontaneous and unique 
human interaction experienced by school leaders requires reflection-in-action and collective inquiry to play 
significant roles in the continuing development of principals - development that is essential in pursuing 
changing organizational culture, practice, and performance through communities of practice or professional 
learning communities (DuFour, DuFour, & Eaker, 2009). 

4.3 Reflective Practice 

There are a diverse set of definitions of reflection ranging from simply thinking about something to engaging 
in a more crafted and strategic practice of looking at a particular problem or phenomenon. Reflection has 
been defined as a thinking process that involves a deliberate pause accompanied by an open perspective 
that examines beliefs, goals, and practices to gain new insights and understanding (York-Barr, Sommers, 
Ghere, & Montie, 2001). Loughran (2002) offered, “it is important that the nature of reflection be identified 
in such a way as to offer ways of questioning taken-for-granted assumptions and encouraging one to see his 
or her practice through others’ eyes” (Loughran, 2002, p. 33). As a strategy for professional adult learning, 
Schon (1987) identified reflective practice as being crucial in overcoming the “technical rationality” (p. 3) 
that pervades the learning in higher education institutions. 

Whatever the definition used, it is perhaps most important to recognize reflective practice as being an 
integral part of the “conceptual characteristic” of the professionalizing process as proposed by Houle (1980). 
According to Houle: 

4.3.1 

It is difficult but necessary to seek constantly to understand the structural tenets of a practitioner’s work 
- those which give it focus and form. Nonexperts often have no trouble defining the central mission of a 
profession.. ..But anybody who thinks about the realities of the professions knows that such definitions are 
too simple to be useful in dealing with the priorities and ethical decisions encountered in routine practice, 
(p. 35) 

Through reflective practice, school principals develop a conceptual understanding of what their job should 
be as opposed to an assumed role that is molded by public opinion or historical perceptions. Never has this 
been more evident than in today’s world of high-stakes accountability where principals are asked to take on 
the role of an instructional leader while continuing to maintain the managerial responsibilities of running a 
school. 

Although the research on reflective practice in school leadership is sparse, Day’s (2000) study of twelve 
highly effective principals found that there were five different kinds of reflection: 

• the holistic , where the emphasis is placed upon vision and culture building; 

• the pedagogical , (on and in action), in which they place emphasis upon staff acquiring, applying and 
monitoring teaching which achieves results allied to their vision (which includes but is greater than the 
demands made by policy implementation imperatives); 

• the interpersonal , where the focus is upon knowing and nurturing staff, children, parents, and governors; 

• the strategic , where the focus is upon entrepreneurship, intelligence gathering and networking to secure 
some control of the future; 

• the intrapersonal , where the focus is upon self-knowledge and self-development and fulfillment. (Day, 

2000, p. 118) 

Day’s five types of reflection suggest that reflection can be used for different purposes ranging from personal 
development to developing a school-wide vision. 

York-Barr et al. (2001) proposed four different experiential contexts that reflection can occur in schools. 
These include individual reflection, reflection with a partner, reflection in small groups or teams, and school¬ 
wide reflection. It can be feasible to match these various contexts within which reflection occurs to the 


http: //cnx.org/content/m44950/1.3/ 


Connexions module: m44950 


6 


multiple types of reflection proposed by Day (2000). For example, in holistic reflective practice, a school¬ 
wide approach might be suitable; in pedagogical and interpersonal reflection practices, reflection with a 
partner might be most suitable; in intrapersonal reflection, individual reflection would be most appropriate, 
and so forth. 

Drago-Severson (2009) identified the following collegial inquiry practices seen by principals as highly 
effective and meaningful to their development: reflection through the process of writing, reflection through 
dialogue, reflection through decision making, and reflection through helping and advising others. It would 
be feasible to match Drago-Severson’s collegial inquiry practices to the types of reflection identified by Day 
(2000) and contexts of reflection identified by York-Barr et al. (2001) as portrayed in Figure 2. 


York-Bmx (2001) Day (2(XX1) 

Rc(lection ilia! is Reflection that is... 

Drago-Severson (2009) 

Rc fleet ton through... 


Holistic 



Helping and 
advising others 


Dialogue 


Decision-making 


Individual 


^ Intrapersonal Writing process 


Figure 2. Identifying the similarities between various types of reflection as presented by York-Barr (2001), 
Day (2000), and Drago-Severson (2009). 

4.4 Peer Learning 

Peer learning, a bi-product of effective collegial inquiry or “collaborative reflective practice” (Drago-Severson, 
2009, p. 154), involves individuals “learning from and with each other in both formal and informal ways” 
(Anderson & Boud, 1996, para. 6). According to Boud (1999), “Peer learning involves a group of people 
taking collective responsibility for identifying their own learning needs and planning how these might be 
addressed” (p. 4). The above definitions of peer learning offer such terms as collaborative , collective respon¬ 
sibility, learning from and with each other. All of these support the premise of self-directed learning and 
social interactions set forth in adult learning theory. Adult learning is enhanced through social interactions 
between individuals that focus on a common problem, and self-directed learning is one of the pillars of adult 
learning theory (Knowles, 1992). When adults are given the opportunity to work collaboratively with little 
intervention from facilitators, self-directed learning occurs (Mezirow, 1997). Participants must value collab¬ 
oration and embrace the opportunity to share perspectives in a safe environment where it is acceptable to 
make mistakes and to seek assistance and clarification when needed (Anderson & Boud, 1996; Boud, 1999). 
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In addition, the learning environment must encourage and nurture reflective dialogue (York-Barr et al., 2001; 
Drago-dSeverson, 2009). 

One form or structure that supports peer learning is the community of practice. According to Hansman 
(2001), “Communities of practice are self-organized and selected groups of people who share a common sense 
of purpose and a desire to learn and know what each other knows” (p. 48). Communities of practice provide 
an environment where competence and experience combine to allow for the acquisition and the creation 
of knowledge (Wenger, 1998). To further distinguish communities of practice, Wenger (2009) suggested, 
“Communities of practice are groups of people who share a concern or a passion for something they do and 
learn how to do it better as they interact regularly” (p. 1). 

A distinction between communities of practice and professional learning communities should be made 
as groups of educators who meet to discuss issues about instruction and learning, continue to learn about 
their craft, and share ideas with others are often referred to as professional learning communities. We see 
professional learning communities as being a type of community of practice with the fundamental difference 
being the communities of practice acquire and generate new knowledge as part of a general social learning 
theory while learning communities tend to concentrate on only acquiring new knowledge (Wenger, 1998). 
Professional learning communities can be seen as a form of professional development implying the acquisition 
of knowledge, while communities of practice can be seen as both professional development and professional 
transformation as knowledge is both acquired and created. 

The Arkansas Leadership Academy is a community of practice that incorporates both reflective practice 
and peer learning networks as significant components to its Master Principal Program (MPP). A major goal 
of the MPP is to develop school leaders who are capable of implementing and managing meaningful change 
within their schools and school systems. 

Supporting the MPP’s community of practice, peer learning networks are in place to accommodate the 
knowledge-building between participating principals. While the phrase peer learning network is most often 
associated with technology-driven online learning opportunities, the ALA defines peer learning networks as 
any opportunity for the participants in the MPP to get together, virtually or physically, to enhance their 
own learning through the interaction with others. 

5 Methods 

Both quantitative and qualitative approaches were used in the analysis of data. In the first phase of the 
study, a quantitative approach was used to address the following hypotheses related to leaders’ change: 

• Hypothesis 1. Principals participating in the ALA MPP will score significantly higher on subsequent 
application portfolios than on their initial application portfolios. 

• Hypothesis 2. Principals who engage in additional RPPLS opportunities will score significantly higher 
on their subsequent application portfolios as compared to their initial application portfolios. 

Through this quantitative approach, the effects of the peer learning networks were examined. The qualitative 
approach addressed the question: 


5.1 

How do the written accounts of the ALA MPP participants represent their reflective thoughts about their 
professional work as a principal? 

5.2 Subjectivity Statement 

The research team was comprised of four members, three of whom had strong associations with the ALA. 
One of those three was directly involved with the Master Principal Program. Because of the involvement of 
the three ALA-affiliated research team members, it was important to recognize that there might be bias to 
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the success of the ALA MPP program. For this reason, this study was not evaluative in nature. Instead, 
the study was designed to examine the reflective process of the program participants and to determine if 
there was a relationship between reflective practice, involvement in peer learning support networks and 
performance on the portfolio assessments. 

The team was aware of the potential for researcher bias and took measures to protect the research process 
as discussed in the methods section; however, it was also recognized that there was value in the subjectivities 
of the researchers as they had an intimate understanding of the Master Principal Program (Maxwell, 2012). 
Therefore, the research team did not identify their potential bias as a threat to the study as long as measures 
were taken to acknowledge their own subjectivities and to approach the study with a sound methodology. 

5.3 Participants 

Study participants were 59 principals from throughout the state of Arkansas who applied for participation 
in MPP Phases 2 and 3 from 2005 through 2010. Principals with portfolio scores for Phase 2 and subsequent 
Phase 3 applications were included in the analysis to allow for repeated measures analysis. Of the 59 
participants, 85% were female and 93% were Caucasian. Urban, suburban and rural schools were represented. 
Out of the 59 participants in the quantitative study, there were 14 portfolios chosen using random purposeful 
sampling as the total number and length of narrative texts available was too large for review for in-depth 
qualitative analysis (Creswell, 2013). 

The 59 principals in this study were the population of principals that completed the MPP through Phase 
3 application between 2005 and 2010 and whose Phase 2 and Phase 3 records could be linked for analysis. 
These participants constitute 91% of the 65 MPP Phase 3 graduates. Given the eligibility requirements for 
participation in MPP, it follows that Phase 3 principals would have a minimum of 4 years of experience as a 
principal. However, there are five principals with less than 4 years. These principals may have been admitted 
to Phase 1 as new principals or assistant principals on a case-by-case basis. The years of experience of the 
participating principals can be found in Table 1. 

Table 1 

Years of Experience as Principal 

Years of experience 2 3 4 5 6 10 12 17 

Percent of study 7.14 10.71 50.00 7.14 10.71 3.75 7.14 3.57 

participants 


5.4 Instrument 

The MPP portfolio is required to be submitted by applicants desiring to participate in MPP Phases 2 and 
3. Phase 2 and 3 portfolios are scored using the MPP rubric. The MPP rubric provides detailed indicators 
which delineate the knowledge, skills and abilities of principals along a 5-point continuum of effectiveness. 
The MPP rubric was designed to differentiate applicants’ readiness for further development as a Master 
Principal. Applicants are scored on the 5-point scale for each of six performance areas resulting in a Total 
Point Score (TPS) out of 60 possible points. Portfolios are scored by ten independent scorers (i.e., two 
scorers for each performance area). The team of scorers were selected from the following groups: 

• Designated master principals; 

• Superintendents and other district administrators; 

• Education leadership faculty from universities and colleges in Arkansas and other states; 

• Arkansas Department of Education leaders from the Learning Services, Accountability and Educator 
Development divisions; 
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• Advocacy organization leaders including Arkansas Education Administrators Association, the Arkansas 
Association for Supervision and Curriculum Development, National and Arkansas School Boards As¬ 
sociation, National and Arkansas Association of Secondary Principals, and National and Arkansas 
Association of Elementary Principals; and 

• Staff members of national and other state educational leadership development programs. 

Team members received rubrics and scoring guidelines two weeks prior to the scoring activity. At the 
beginning of the two-day scoring session, there was a guided practice session which led scorers through a 
sample portfolio assessment. Through this process,inter-rater reliability was established prior to independent 
scoring of the portfolios. 

While participants in the MPP submitted school performance data as part of their portfolio, the narratives 
were written describing their school, the relationship of their school and community, their own practice as 
principal related to the six performance areas of the rubric, and the overall progress of their school. 

5.5 Research Design 

Two repeated measures analyses of ALA MPP principals’ portfolio scores were conducted to test the afore¬ 
mentioned hypotheses: 

1. A paired-samples t -test of principals’ application portfolio scores was conducted using Phase 2 appli¬ 
cation portfolios and Phase 3 application portfolios as initial and subsequent measures of principals’ 
leadership knowledge and skills, and 

2. A one-between repeated measures analysis of variance (two-way repeated measures ANOVA) was con¬ 
ducted using principals’ application portfolio scores for Phase 2 and 3 portfolios with additional Insti¬ 
tute Participation as the between factor. 

Repeated measures designs provide more power for detecting meaningful differences because idiosyncratic 
differences are removed from the error term (Stevens, 2002). The increased power of repeated measures is 
particularly important in cases where small sample sizes are unavoidable. 

For the two-way repeated measures ANOVA principals were grouped into three categories based on the 
number of additional ALA Institutes that principals participated in prior to or during participation in Phase 
2 of the MPP. These ALA institutes focused on developing various school improvement related knowledge 
and skills through the use of the RPPLS learning approach. The categories were classified using the following 
frequencies. 

• Category 1-participation in one additional RPPLS Institute (27 participants) 

• Category 2-participation in 2 to 4 additional RPPLS Institutes (27 participants) 

• Category 3-participation in 5 or more additional RPPLS Institutes (5 participants) 

Following the quantitative analysis, a qualitative analysis was conducted of Phase 2 and Phase 3 applicants’ 
narratives provided in their portfolios. An informal approach to the analysis of the written narratives was 
used. Per"kyl" (2008) explained: 


5.5.1 

In many cases, qualitative researchers who use written texts as their materials do not try to follow any 
predefined protocol in executing their analysis. By reading and rereading their empirical materials, they 
try to pin down their key themes and, thereby, to draw a picture of the presuppositions and meaning that 
constitute the cultural world of which the textual material is a specimen, (p.352) 

Per"kyl" (2008) also suggests that an informal approach to analyzing text is most often seen as an ac¬ 
ceptable way to process narrative writing when the textual analysis is seen as a complementary or supporting 
component of the larger research design. The goal of the analysis was to discover any change in the written 


http: //cnx.org/content/m44950/1.3/ 


Connexions module: m44950 


10 


reflections of the program participants with the intent to determine if the level of reflection by the par¬ 
ticipants changed between the writing of the Phase II portfolio narrative and the writing of the Phase III 
portfolio narrative. 

The portfolio exercise was designed, in part, as a reflective writing exercise where MP candidates were 
asked to reflect on their experiences as a school leader in relation to the principles and standards of the MP 
program. Given that the data was pre-existing, and not collected specifically with the intent of uncovering 
the reflective practices of the participants, it can be determined that more work might be done to gain a 
better understanding of the reflective process experienced by participants in the ALA MPP. 

Phase II and Phase III portfolios produced between 2008 and 2010 were read. A post-positivist approach 
(Creswell, 2013) to the analysis initially involved a deductive analytical approach using the theories presented 
in Figure 2. The first cycle of descriptive coding (Saldana, 2009) led to the realization that emerging themes 
could be categorized using van Manen’s (1995) temporal dimensions (see Table 2). Therefore, a second cycle 
of deductive coding was used to further reduce the data and further define emerging themes. 

Table 2 

Temporal Dimensions of Reflection 


Temporal dimension 

Definition 

Retrospective 

Contemporaneous 

Anticipatory 

Reflection on past experiences (Reflection on action) 

Reflection while acting (Reflection in action) 

Reflection on future experiences 


The data analysis approach was conducted by a one member of the research team. Once the coding 
was done and themes identified, the analysis was vetted to the other members of the team through peer 
debriefing to establish credibility to the analysis (Marshall & Rossman, 2011). This approach was used to 
reduce researcher bias during the qualitative analysis as the researcher who actually conducted the initial 
analysis was not associated with the ALA. 

While retrospective and anticipatory dimensions are the most common, contemporaneous reflection sug¬ 
gests the most powerful type of reflection (Sch"n, 1983; van Manen, 1995). It is through contemporaneous 
reflection or reflection-in-action that practitioners surpass the simple application of theory or what Sch“n 
would define as technical rationality. Contemporaneous reflection seizes the moment and allows for a deep¬ 
ening of awareness that is driven by the context of the present situation the practitioner finds themselves. 
Raelin (2001) explains: 


5.5.2 

Contemporaneous reflection occurs in the moment, akin to Schon’s (1983) Teflection-in-action’, such that in 
the midst of performance one reframes unanticipated problem situations in order to see experience differently. 
While engaged in experience, planned responses often don’t go according to plan, triggering a series of 
unexpected reactions. In this situation, the learner often reframes the problem on the spot in order to 
release oneself as well as one’s colleagues from fixed views, leading to the consideration of new approaches. 
(Raelin, 2001, p. 19) 

It is the contemporaneous reflection that is the most difficult to identify, particularly when using existing 
data from narratives that were written for purposes other than current study. However, by pursuing the 
identification of contemporaneous reflection in the narratives, the findings illuminated the thought that while 
reflection-in-action was difficult to determine, the shift to more holistic reflection was evident. 
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6 Results 

6.1 Quantitative Analyses 

Results of the dependent samples t -test indicated principals applying to Phase 3 of the MPP scored signif¬ 
icantly higher on their Phase 3 portfolios than their Phase 2 portfolios (t = 8.12, p < 0.0001). Principals 
scored 37.14 (cr = 5.46) points on average for Phase 3 application portfolios compared to 30.20 (a = 6.12) for 
Phase 2 portfolios. The effect represents a large or meaningful difference (d = 1.06) in principals’ knowledge 
and skills from the beginning of Phase 2 to the beginning of Phase 3 MPP. 

The two-way repeated measures ANOVA with portfolio scores as the repeated measure resulted in a 
significant interaction between principals’ frequency of participation in RPPLS and the change in portfolio 
scores from Phase 2 to Phase 3 applications (F( 2 , 56 ) = 4.22, p = 0.02). Table 3 provides the summary table 
for the two-way repeated measures ANOVA. 

Table 3 

Summary Table for the Two-Way Repeated Measures ANOVA 


Source 

4£ 

SS 

MS 

F 

P 

Between Subjects 

58 

2647.04 




Events 

2 

766.50 

383.25 

11.41 

<0.0001 

Residual between 

56 

1880.54 

33.58 



Within Subjects 

58 

2230.89 




Phase 

1 

978.64 

978.64 

50.36 

<0,0001 

Events 11 ' Phase 

2 

164.06 

82.03 

4.22 

0.02 

Residual within 

56 

1088.19 

19.43 



Total 

58 

4877.93 





Figure 3 illustrates the interaction between the number of RPPLS Institutes attended by principals and 
the change in portfolio scores from Phase 2 to Phase 3 applications. Principals involved in Category 3, 5 
or more RPPLS Institutes by the completion of Phase 2 MPP, achieved the highest mean portfolio scores; 
whereas principals attending fewer than 5 ALA Institutes demonstrated lower average performance by the 
end of Phase 2 MPP. 
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Change in Mean Portfolio Scores 
for Principals by Institute 
Participation 



* Attended I Institute 
—A? Attended 2 - A Institutes 

Attended more than 4 Institutes 


Figure 3. Mean rubrics scores of participants’ portfolio assessment in each of the three categories for 
Phase 2 and Phase 3. 

6.2 Qualitative Analyses 

The voices of the participants suggested that Master Principals had developed into more holistic reflective 
practitioners as they moved from Phase II to Phase III of the MPP. The narratives of the Phase II portfo¬ 
lios presented rich data, the reflections included almost exclusively retrospective and anticipatory reflective 
thought. There were more instances of broad realizations of their professional experience as a principal 
displayed in the Phase III portfolio reflections. 

Many of the participants’ reflections in the Phase II portfolios could be described as being reports of 
actions taken in the past (i.e, retrospective reflection). For example, one elementary principal wrote: 


6.2.1 

We also decided to change from having faculty meetings every week. When possible, we asked content area 
teachers to meet together as teams for Science, Math, Literacy, or Social Studies. In team meetings we were 
able share ideas and to have conversations about strategies and student learning. 

This “look into the past” is a form of retrospective reflection that might help this particular principal 
identify where her school was at one point in time. By doing so, she may shed light on how much the 
organization has changed its meeting practices, but there is little to suggest how her practice as principal 
has changed. 

Anticipatory reflection was also evident in the Phase II portfolio narratives. This same participant offered, 
“Once new vision is established, we will work to align our policies and procedures to our core beliefs.” Here, 
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the principal gives some ideas of what she is envisioning is going to happen in the future related to extending 
the school’s vision. One participant anticipated the need to continue working on instructional issues: 


6.2.2 

The journey towards more effective teaching and learning has only just begun. We will continue to learn and 
grow as we incorporate book studies on effective strategies, common assessments, responses to intervention, 
and becoming PLCs at our team meetings over the summer and next year. 

Yet another Phase II participant exhibited anticipatory reflection about instructional matters through 
the statement: 


6.2.3 

There is no time to sit back and rest. We all know that on August 21 we start all over again with a different 

set of students. The challenge is the ever-moving target that No Child Left Behind creates_It won’t be 

until every child reaches proficiency that we’ll be satisfied. 

The anticipatory reflections of the Phase II participants represent planning based on current or prior 
experience and projecting what lies in store in the future (Raelin, 2001, van Manen, 1995). 

In the Phase III portfolio narrative, these same principals revealed anticipatory and retrospective reflec¬ 
tions - much like the reflections in their Phase II portfolios. However, there were instances of a more holistic 
reflection. These reflective statements might not necessarily be identified as contemporaneous reflections; 
however, they do have characteristics of intrapersonal and holistic reflection as described by Day (2000). 

In the Phase III portfolio, this one participant wrote about her school’s growth in sharing a vision and 
mission: 


6.2.4 

In recent years, I have realized the importance of a culture in which all members share a belief system about 
student learning, a common vision and mission, and an acceptance of the responsibility to help create an 
environment in which all of our students will thrive. We have made progress in this area. 

The phrase “I have realized...” implies an intrapersonal and holistic reflective thought - one of realization 
not only of the importance of a culture that has a cohesive belief system, but also one of the realization 
of progress being made. Another participant stated that she had come to the realization that “We [the 
faculty] had turned into a family...” while another expressed, “we are never satisfied with the status quo.” 
The statement, “I realize now that I have been an agent of change!” illuminates an intrapersonal reflective 
thought (Day, 2000). The data from all analyzed portfolios generally followed the trend found in Figure 4. 


Phase II 

Phase III 

Ret rospect i ve 

RlH rospect i ve 
Reflection 

Reflection 


K Anticipatory 

Anticipiilory 

lie flection 

Reflection 

Holistic/ 

Inf rape rsona l 


Reflection 


Figure 4. The trend of participants’ reflective thoughts from retrospective and anticipatory reflection to 
a more robust and reflective process with the addition of holistic and intrapersonal reflective thought. 
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7 Discussion 

The findings supported a connection between peer learning support networks and effective development of 
school leaders. This study also suggested that Master Principal candidates’ reflection process becomes more 
holistic and intrapersonal in nature as they move through the various phases of the program. Quantitative 
findings suggested that more opportunities for structured reflective practice and peer learning are associated 
with significantly higher scores in leaders’ reflective practice as measured by their narrative writings. The 
qualitative findings specified the nature of change in leaders’ reflective writings with the addition of holistic 
and intrapersonal reflection enhancing retrospective and anticipatory reflections. 

This study suggested that there very well could be a correlation between the types of development that 
school leaders participate in once they are in the seat and their development as reflective practitioners. Given 
that organizational change in culture, practice and performance are primary pursuits of school leaders, there 
is value in determining the impact of professional learning practices such as RPPLS on continued leader 
development in these areas. We have established that these participants have shared similar experiences as 
members of peer learning support networks which emphasize reflective practice through various professional 
development opportunities that promote reflective thinking and action. However, this study left the question 
of whether the changes in leaders’ reflective practices are associated with changes in student performance 
unanswered. While there is a suggested correlation between Master Principal candidates’ experience in the 
program with the increased performance of their schools, we cannot legitimately say that there is a causal 
relationship between participation in the Master Principal Program and school performance. 

There were limitations to this study. The most noticeable being the use of extant qualitative data that 
was obtained from the ALA archive. While the preexisting portfolios did provide an opportunity to examine 
a “longitudinal” aspect to the written reflections of the participants, we were limited by the constraints of 
the requirements of the portfolio assignment. We feel strongly that the use of extant data and not having 
the opportunity to collect “fresh” data limited us in determining the level of reflection-in-action that actually 
might have taken place in the daily practice of the participating principals. 

An additional limitation that we accept was rooted in the realization that to find any direct causation 
between school leadership and student achievement remains unexplored. Principals who have been involved in 
the Arkansas Leadership Academy Master Principal Program have self-reported, in general, positive growth 
in their schools’ performance as measured by student achievement data. Given the concerns and limitations 
of self-report, a follow-up study is underway to analyze whether changes in leaders’ reflective practice are 
associated with changes in student performance over time. The relationship that should be considered based 
on the results of this initial study is the role of the ALA MPP with its emphasis on peer learning networks 
and reflective practice and the positive development of leadership. 

Given these limitations, this study provided an examination of ongoing professional development for 
principals and the development of reflective practice through peer support networks. While the existing 
literature identifies specific types and contexts of reflection found in school leaders (Day, 2000; Drago- 
Severson, 2009; York-Barr et ah, 2001), there has not been a prior examination of the change in reflective 
practice due to involvement in an ongoing leadership development program. Additionally, there has been 
past research done on peer learning support networks (Anderson & Boud, 1996; Boud, 1999; Drago-Severson, 
2009) and communities of practice (Hansman, 2001; Wenger, 1998, 2009); however, there has not been an 
examination that links peer support learning networks with the development of reflective practice. This study 
was a first step in establishing that reflective practice, as defined by the literature cited, can be developed 
through an ongoing leadership development program such as the Arkansas Leadership Academy’s Master 
Principal Program. 

This study has implications for formal university leadership preparation programs by suggesting that peer 
learning and reflective practice supports the development of leadership. Caution, however, must be used in 
suggesting that the MPP is identical to a leadership preparation program. The participants in the MPP were 
already engaged in the principalship and were continuing their development , while university preparation 
programs generally prepare pre-service school leaders. Recognizing this difference and understanding that 
professional preparation and professional development both should be anchored in adult learning theory, the 
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findings do indicate that an intentional effort to promote adult learning theory through reflective practice 
and peer learning should be considered applicable to preparation programs. 

Further implications include the broadening the understanding and recognition of the difference between 
knowledge-in-action and reflection-in-action and how reflective practice (both collaboratively and individu¬ 
ally) can benefit the performance of school leaders as they manage, facilitate, and lead change initiatives in 
their schools and systems. School systems should promote school leaders’ involvement in reflective environ¬ 
ments where there is ample opportunities to share knowledge with and learn from others who are actively 
engaged in the business of leading schools. This is not as easily done through formal preparation programs 
as many educational leadership students are not practicing as school leaders. Therefore, it is important that 
those institutions providing formal leadership preparation programs understand and value those indepen¬ 
dent organizations that allow for the continuing development and learning of principals. There is a need for 
research to inform the practice of continuing education of educational leaders and as mentioned above, such 
research could also help inform leadership preparation programs. 

8 Conclusion 

Principals gain knowledge and skills through the ALA MPP which provides participants with a rich learning 
environment through reflective practice and peer-learning supports embedded in the MPP and ALA Insti¬ 
tutes curricula. Notably, principals who participate in more opportunities for RPPLS demonstrate greater 
leadership knowledge and skills as measured by MPP rubric scores. The repeated measures analyses provide 
ample power to detect meaningful differences. However as with any single group design, more information 
is needed to compare the changes observed in MPP principals with other principals who did not participate 
in the MPP leadership development program. 

The promotion and nurturing of relationships between university preparation programs and organizations 
like the Arkansas Leadership Academy should be of concern for all involved. No longer can programs like the 
Arkansas Leadership Academy be seen as the “competitor” working against formal university preparations 
programs. In Arkansas, it is believed that the Arkansas Leadership Academy is significant piece to the puzzle 
of leadership performance throughout the state and the lessons learned through the study of the Academy’s 
various programs can help illuminate the challenges and possible solutions to the continuing education of our 
practicing educational leaders as they seek positive change in their personal, school, and system performance. 
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Appendix A 

Arkansas Leadership Academy Master Principal Program Rubric Sample 2 

Appendix B 

Arkansas Leadership Academy Master Principal Program Conceptual Scoring Framework Sample 3 


2 See the file at <http://cnx.org/content/m44950/latest/Appendix A.pdf> 

3 See the file at <http://cnx.org/content/m44950/latest/Appendix B.pdf> 
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